Formulas

Elementary Matrices

• Inverse of Row Swap:
• Inverse of Row Scaling:
• Inverse of Row Addition:
• Determinant of Row Swap:
• Determinant of Row Scaling:
• Determinant of Row Addition:
• Invertible Matrix Factorization: , so

LU Decomposition

• LU Decomposition: ( unit lower triangular, upper triangular)
• Existence Condition: All leading principal minors
• LU with Partial Pivoting: (exists for any invertible )
• LU with Complete Pivoting: (exists for any matrix )
• LU Factorization Cost: flops
• Forward/Back Substitution Cost: flops per solve

Triangular System Solving

• Forward Substitution:
• Back Substitution:

LDU and Special Decompositions

• LDU Decomposition: where ,
• Symmetric LDU: (when )
• Cholesky Decomposition: (when is symmetric positive definite, has positive diagonal)
• Cholesky Cost: flops

Positive Definiteness

• Sylvester’s Criterion: Symmetric is positive definite iff all leading principal minors

Determinant with Pivoting

• Row Swap Effect:
• Column Swap Effect:
• From :

Matrix Inversion

• 2x2 Inverse:
• Transpose of Inverse:

Inner Products and Norms

• Dot Product (Inner Product in ):
• Vector Magnitude (Norm):
• Unit Vector: (for )
• Orthogonality Condition: if and only if

Orthogonal Projections

• Projection of onto :
• Orthogonal Component: is orthogonal to
• Orthonormal Set Property:

Gram-Schmidt Orthogonalization

• General Step ():
• First Vector: ,
• Second Vector: ,

QR Decomposition

• QR Factorization: where (orthonormal columns) and is upper triangular
• Computing R:
• Dimension of Q: matrix ( rows, orthonormal columns)
• Dimension of R: upper triangular matrix with positive diagonal entries

Subspaces and Spaces

• Column Space (): Span of columns of ; subspace of for
• Null Space (): ; subspace of
• Complete Solution: where is a particular solution and

Least Squares

• Normal Equations:
• Least-Squares Solution (full column rank):
• Pseudoinverse:
• QR Least-Squares System: (solved by back substitution)

Projection Matrices

• Projection onto :
• Projection Matrix: (projects onto )
• Projection onto a Line (spanned by ):
• Projection Matrix Properties: (idempotent), (symmetric)
• Complementary Projection: projects onto

Eigenvalues and Eigenvectors

• Eigenvalue-Eigenvector Relationship: (defining equation)
• Characteristic Equation: (equation whose roots are eigenvalues)
• Finding Eigenvectors: (solve this homogeneous system for each eigenvalue)
• Trace Property: (sum of eigenvalues equals trace)
• Determinant Property: (product of eigenvalues equals determinant)
• Diagonalization: where (columns are eigenvectors) and (diagonal matrix of eigenvalues)
• Powers via Diagonalization: where
• Inverse via Diagonalization: where (valid if is invertible)
• Eigenvalues of Triangular Matrices: If is upper or lower triangular, then eigenvalues are the diagonal entries
• Eigenvalue of Matrix Power: If is an eigenvalue of , then is an eigenvalue of
• Eigenvalue of Inverse: If is an eigenvalue of an invertible matrix , then is an eigenvalue of
• Linear Independence of Eigenvectors: Eigenvectors corresponding to distinct eigenvalues are linearly independent
• Cayley-Hamilton Theorem: where is the characteristic polynomial
• Spectral Mapping Theorem: If has eigenvalues and is a polynomial, then has eigenvalues
• Eigenvalues of Symmetric Matrices: If (real symmetric), then all eigenvalues are real numbers
• Orthogonality of Eigenvectors: For real symmetric , eigenvectors corresponding to distinct eigenvalues are orthogonal
• Spectral Decomposition (Symmetric): where is orthogonal and is diagonal (eigenvalues on diagonal, columns of are orthonormal eigenvectors)

Similar Matrices

• Similarity Definition: for some invertible matrix ; is similar to
• Powers of Similar Matrix: If , then
• Invariants Under Similarity: Similar matrices share the same determinant, trace, characteristic polynomial, and eigenvalues
• Similarity and Diagonalization: is diagonalizable iff is similar to a diagonal matrix
• Simultaneous Diagonalization: Matrices and are simultaneously diagonalizable iff (they commute) and both are individually diagonalizable

Complex Numbers

• Standard Form: where , ,
• Complex Conjugate: ; properties: , ,
• Modulus:
• Polar Form: where ,
• Euler’s Formula:
• Product in Polar Form: (multiply moduli, add arguments)
• De Moivre’s Formula:

Complex Inner Product

• Hermitian Inner Product in :
• Norm in :
• Conjugate Symmetry:
• Positive Definiteness: , with equality iff
• Cauchy-Schwarz Inequality:

Conjugate Transpose

• Definition: ; equivalently
• Properties: , , ,
• Inner Product via Matrix Multiply:

Hermitian and Unitary Matrices

• Hermitian Matrix: (generalizes real symmetric matrices to )
• Hermitian Eigenvalues: All eigenvalues of a Hermitian matrix are real
• Hermitian Eigenvectors: Eigenvectors of a Hermitian matrix for distinct eigenvalues are orthogonal (w.r.t. Hermitian inner product)
• Unitary Matrix: (equivalently ; generalizes real orthogonal matrices)
• Unitary Preservation: and for all
• Unitary Eigenvalues: All eigenvalues of a unitary matrix lie on the unit circle ()
• Unitary Determinant:
• Normal Matrix: (includes Hermitian and unitary matrices as special cases)
• Spectral Theorem (Complex): Every Hermitian matrix (and more generally every normal matrix) is unitarily diagonalizable: where is unitary and is diagonal with real entries (for Hermitian )
• Spectral Decomposition: where are orthonormal eigenvectors
• Four Fundamental Subspaces (Complex): , , , ; uses in place of